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ABSTRACT 

The question of whether performance measures may be 
more accurate than examination methods to assess an individuals 
abilities and knowledge is examined, based on a literature review. An 
information processing view that focuses on the internal activities 
of thinking and the relationship to external stimuli is considered. 
Information processing studies indicate that: (1) a person applies 
one's own rules and strategies in problem-solving, and (2) 
comprehensive test is a general test that does not consider the fact 
that some individuals may require more information or time 
sufficiently answejr the question. Attention is directed to 
control processes/" or mechanisms that vary among 
that determine Jthe learner's information processing approaches for 
different learning tasks. The functioning of the executive control 
process in a problem-solving activity is related to progressive 
deepening and/ react ion time . Progressive deepening is a proceis that 
humans go thrbugh as problem-solvers in a hypothetical action^ It is 
proposed Jthatythe evaluation of individuals exposed 
pert ormance-orl^ea^ed instruction should 
performance orientation, flexible time 
use one's own strategies. (SW) 



"executive 
individuals! and 



to 

include attributes 01 
constraints , and f reec 



the 

lorn to 



/ 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



9 

ERIC 



THE CASE AGAINST ^HE COMPREHENSIVE EXAM 



" by 1 Jon D. Morris 

Instructor, Dept. of Mass Communication 
University of Louisville 

Doctoral student in Curriculum and 
Instruction at the University of Florida 



US. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 

EDUCATIONAL RESOURCES INFORMATION 
y CENTER IERICJ * 

CKThis document has been reproduced as 
received from the person or organisation 
originating it 

Minor changes have been made to improve 
reproduct«on quality 

• Pomts of viewv o» opinions stated in this docu 
ment do not necessarily represent official NlE 
position or policy 



■ PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) ■ 



INTRODUCTION 

The study of learning goes b^ck at least 100 years. (Kulik and 
Kulik, 1979). Ebbinghouse, Thorndike, Pavlow, and Watson, among others, 
pipneered methods for examining the relationship of thinking to learning. 
Early studies indicated that learning was predominately an external activity; 
that is, a given stimulus was said to produce a given response with little 

consideration given to the organism through- which this process must pass. 

i. ,■ 

No matter the background, influences or composition of the organism, stimuli 
were said to produce expected 'responses. Unexpected responses wete attributed 
to deficiencies in the organism itself. 

( 

More recently, educational research and theory has focused on the 
internal activities of thinking and the relationship to external stimuli. 
From this focus, educators have learned about the unanticipated ^responses 
to stimuli and, more importantly, the significance o^Ridividual differences. 
Researchers now believe that an unanticipated response to a particular stimulus 
may be attributed to the way an organism perceives the stimulus father than with 
any deficiency in the organism itself. 

Directly translated into educational practice, this focus on human under- 
standing and thinking, known as "information processing". challenges the 
accuracy of standardized evaluation in certain instances. Does an examination, 
for example, measure an individuals' true knowledge ^and abilities or are there 
other important factors to be considered? Is it possible that inadequate 
responses to examination questions may be attributable to lack of response 
time, or varying perceptions of what the question is asking. Given ttie diff- 
fences in individuals cited by the recent studies in information processing, 
would performance methods of evaluation be more accurate for some individuals 
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than comprehensive examinations? 

The section that follows, is a review of the literature as it relates 
tp the above questions, and is an attempt to show that, in fact, performance 
measures may be more accurate than examination methods when assessing an 
individuals abilities and knowledge, 
PERFORMANCE MEASURES VS. THE COMPREHENSIVE EXAM 

Concerns by educators as to the accuracy of the comprehensive exam- 
ination as a measure of a, student's knowledge and ability are not new. In 
1933, Jones examined the comprehensive examination in American colleges, Ii\ 
his chapter titled "Improving the Examination 11 , Jones cites an even earlier 
1918 English analysis of the written comprehension examination process and 
its relationship to culture. According to Jones, Harfcog, the edticator, de- 
plored the "tremendous number of failures in the great mass of examinations in 
the United Kingdom — about 50 percent. This is partly because written examining 
is so mechanical. It is geared for the masses and cannot* fit the individual". 
(Jones, 1933, p. 221), 

Jones added that many superior college students indicated they would not 
want to stake everything on such an examination. Discontentment with this 
process has continued through the decades, but research has offered little sup- 
port for these misgivings about the adequacy of the comprehensive exam, until 
the development of the information processing models of educational theory. 

Recent developments in human information retrieval and comprehension have 
led psychologists Lindsay and Norman (1972) to conclude that "the development 
of individual differences and idiosyncratic systems should be the rule, rather 
than the exception. Understanding evolves through a combination of external 
evidence and the internal operations that manipulate and reorganize the incoming 
information". (p. 432) 



Why does this recent information about how an individual thinks pose 
serious questions about the comprehensive exam as a valid measure for everyone? 

To understand the need for performance measures rather than descriptive 
measures in particular instances, one 'must examine the "vigorous study of 
cognition and individual differences within an information-processing frame- 
work," (Doyle, 1979, p. 183). "The current fqcus, in other words, is not simply 
oft the supportive atmosphere of the classroom; it is also on the activities that 
Occupy students' time during instruction". (Doyle, 1979, p. 185) 

Performance can be an important part of learning as well as an important 
tool for" the assessment of an individual's ability to apply this knowledge in 
a problem solving situation. 

"Knowing the recipe for preparing food does not, as many novice knows, 
assurethe success of the finished product. Extensive knowledge of vocabulary 
and rules of grammar do not, in themselves, assure a student of the ability to 
express himself and his ideas in literary endeavors. It is for these reasons 
that performance tests are sometimes important devices for assessing educational 
achievements". (Ryan and Frederiksen, 1951, pp. 455-456) 

Information processing studies according to Gagne (1977) have attempted to 
construct a framework for explaining human understanding. The relationship of 
performance to theories of information processing is important to the tield of 
. curriculum and instruction because C & I is responsible for translating these 

theories into practice. 

Today's living calls for problem-solving skills, concept formation, 

4 

data-processing skills, the ability to make judgements and discriminate, 
the ability to relate causes to effects* the ability to analyze, the 
ability to summarize and the ability to form valid conclusions. In summary, 
' present day education places too much emphasis on the learner's memorization 
t 
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of information. Problem solving skills are neglected. (Burns 
and Brooks, 1974, p.' 42-43) 
From Information Processing Studies, five important findings have 
surfaced which underscore the need for including performance measures when 
planning curriculum and instruction. 

1) Although there is a lack of agreed upon tools for analyzing individ- 
ual exploratory behaviors, protocal suggests that a person gradually 
accumulates new information about a problem by applying his or her 
own rules and strategies. (Lindsay and Norman, 1972) These individ- 
ual rules and strategies may be foreshadowed by the rules and 
strategies permitted by the comprehensive examination. A conflict 
occurs then between the rules of the exam and rules and strategies 
of the individual student. A performance^ measure may neutralize 
these differences in rules. 

2) The method by which the student learns and the method by which 
student is examined should also be similar. If performance methods 
are used as the principle tools for learning they the^i should also 
be used as the methods of evaluation* 

3) Even though many comprehensive examinations attempt to measure 
problem-solying skills and abilities as well as knowledge, exam- 
inations cannot, in all cases function adequately because of the 
differences in individual learning mechanisms. These mechanisms that 
vary from individual to individual are often labeled "executive 
control processes". "The function of the executive control process 
is to determine the particular kinds of information processing in 
which the learner engages to accomplish particular kinds of learning 
tasks/ They determine the learner's approach to one or more ways of 

v processing information". (Gafene, 1977, p. 59) 
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Executive control strategies_vary from individual to individual and 
from task to task. There are two variables, however, that -are related 
directly to the functioning of the 'executive control process' in a prob- 
lem solving activity. One is the method of "progressive deepening ". The 
other is "reaction time" . 

4 

A) PROGRESSIVE DEEPENING 

Progressive deepening , according to Newell and Simon, is a process that 
humans go through as problem solvers in a hypothetical action.* (Eisenstadt 
and Kareev, 1975) "Progressive deepening is a strategy based on a decision 
tree/ Specifically, they (learners) pursue a branch until a decision can 
be tested. Once it has been tested and found lacking, the problem solver 
returns to the base and pursues another branch". (Davis, Alexander and 
Yelon, 1974, p. 267). 

In order to accomplish "progressive deepening" the performance oriented 
individual must have at his or her disposal tools for investigation and 
analysis. The ability to conduct research, to examine public reaction, and 
to determine the feasibility within the current system are but a few of the 
important aspects of the progressive deepening process in a performance orient 
ation. In other words, the performance process of "progressive deepening" 
relies heavily on feedback. Feedback that is not available in the comprehen- 
sive examination process. "Sensitivity to feedback is a major determinant 
of a person's potential to improve his or her work." (Sternberg, 1981, p. 19) 
This is vital in a performance orientation. 

The "orientation toward intellectual skills, toward what the student is 
able to perform, rather than what he knows r is an extremely important result 
of developing educational technology. It has 'been the systematic development 
of procedures and techniques of instruction, based on psychological theory". 



(Gagne, 1974, pp. 58-59) 
B) REACTION TIME 

The tools for evaluating learning competencies in curriculum and 
instruction design using the process-performance oriented method should 
be process-performance in nature rather than in the form of a comprehen- 
sive time limiting examination. Enter then the second important variable 
in the executive control process in problem solving activities, tim^i Pan- 
chella (1974) says that 'reaction time 1 is an important aspect of/speed- 
accuracy realtionships. This 'reaction time' is credited with/producing two 
types ^of responses; stimulus controlled response and fast-gufess response. 
(Panchella, 1974, p. 70) „ / 

Stimulus controlled responses allow the subject to take as much time 
as necessary for an accurate response. This is indicative of the performance 
process which allows the subject to control his or hjgyp response time, and 
thereby control the response. 

Fast guess responses occur in situation which require cognitive adjust- 
ments to speed stress. For some the comprehensive exam is indicative of this. 
These cognitive adjustijaents may 4 involve the alteration of the critical values 
of the stimulus evidence which must be accumulated. In other words, the quan- 
tifcy and quality of input may be reduced, because the evaluation of such input 
has been removed from the subjects control. This, in turn, will lead to low 
critical values, which will lead to fast reaction times and thereby produce 
high error rates. (Panchella, 1974) 

CONCLUSION ^ 

After fifteen years of study, Simon and Newell, whose information procesy 
sing models inspired the psychological studies of Lindsay, Norman, and Rummal- 
harts; influenced the writing of Gagne, and led to the curriculum development 
models of Eisner and others; concluded: "A few, and only a few of the gross 
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characteristics of the human information-processing systems are invarient 
over task and problem". (Hunt and Poltrock, 1974, p. 344) 

Tasks and problems are as varied as the skills implemented to achieve 
a solution. * "Ability tests seem to provide only the most limited measure- 
ment of a. /...skill". (Sternberg, 1981, p. 1*0 This may be due in 

part to the fact that a comprehensive test is a general test. It does 
not consider the fact that some individuals may require more information 
or more time to sufficiently answer the question. Again, this is not due 
to inability but strictly to an individual's process methods. 

What se^ms^to me to be important lessons to have cbme from program- 
med instruction is, very simply, that instruction must be designed to 
teach the student the capability of doing something not knowing something. 
The notion of the performance objective is important because it emphasizes 
the doing. To use other t.erms familiar to curriculum designers, the primary 
purpose of instruction is process not content. (Gagne, 1974, p. 58) 

But programmed or performance: oriented instruction's greatest strength 
is that it can accomodate individual executive control strategies. That is, 
programmed instruction is flexible with regard to completion time and thereby 
flexible enough to allow an individual to implement his or her own methods of 
•progressive deepending*. 

It seems appropriate then that performance oriented methods should be 
Used in evaluating the abilities of a person that has chosen a performance 
route in instruction. These evaluation methods should in turn include that 
attributes of the performance orientation; flexible time constraints and 
freedom to use ones own strategies. 

According fco Scriven, finding the value or quality of an educational* 
product is a complex matter based upon various kinds of information. Of his 
thirteen suggestions for assessing the value of educational programs or product 



however, nine are concerned primarily with some sort of performance. (Gagne 
and Briggs, 1979, pp. 287-88) 

0 

♦ 

t 



: f < 



ERIC 



8 



10 



BIBLIOGRAPHy 



Burns, R.W., Brooks, G.D. "Processes, Problem Solving, and CurrictfLum 

Reform". In Eisner, E.W. , Wallance, E. (Ed.) Berkeley: Conflicting 
Conception of Curriculum . McCutchan Publishing, 1974* 

Davis, R.H. , Behan, R.A. , "Evaluating System performance in Simulated 

Environment." In Gagne, R.M. (Ed,) Psychological Principles in System 
Development . New York: Holt, Rinehart, and Winston, 196?. 

Doyle, W. "Classrooms Tasks and Students 1 Abilities". In Peterson, P.L. 
& Walberg, H.J. (Ed.) Research on Teaching , Berkeley, California: 
McCutchan Publishing, 1979. 

Eisenstadt, M. and Kareen, Y. "Aspects of Human Problem Solving: The Use of 
Internal Representation". In Norman, D.A. Rummelhart, D.E. (Eds.) 
Explorations in Cognition . San Francisco; W.H. Freeman and Company, 1975. 

Evans, C, The Micro Millenium . New York: The Viking Press, 1979. 

r ' 

Gagne, R.M. The Conditions of Learning . New York: Holt, Rinehart and 
Winston, 1977. 

Gagne, R,M. , "Educational Technology as Techniques". In Eisner, E.W., 

Vallance, E. (Ed.) Conflicting Conceptions of Curriculum . Berkeley, 
California: McCutchin Publishiag Company, 1974. 

Gagne, R.M., Briggs, L.T., Principles of Instructional Design . New York: 

* Holt, Rinehart and- Winston, 1979. 

i 

Hunt, E.„ Poltrock, S. "The Mechanics of Thought". In Kantowitz, B. 

(Ed. ) Human Information Processing : Tutorials in Performance and Cog- 
nition . Hillsdale, New Jersey: Lawrence Erlbaum Associates. 

Jones, E.S. 1933. Comprehensive Exam in Action in American Colleges . New York 
The Macmillian Company, 1933 • 

Joyce, B.i Weil, M. Models of Teaching . Englewood Cliffs, New Jersey: 
Prenticfe-Hall, 1980. t 

Kulik; J. A. , Kulik, C.C.' "College Teaching" . In Peterson, P.L. and Walberg, 
H.J. (Ed.) Research on Teaching . Berkeley; McCutchan Publishing, 1979. 

Lindsay, P. H. , Norman, D.A. , Human Information Processing . New York; 
Academic Press, 1972. 

Panchella, R.G. "Reaction Time' 1 . In Kantowitz, B. (Ed.) Human Information 
Processing : Tutorials in Performance and Cognition . Hillsdale, New * 

r Jersey: Lawrence Erlbaum Asociates, 1974. 

.» • 

Popham, W.J. Evaluating Instruction . Englewood Cliffs, New Jersey: Prentice 
Hall, 1973. 

Rumelhart, D.E., Norman, D.A. , "The Computer Implantation". .In Norman, 

D.A. , Rummelhart, D.E. (Ed.) Explorations In Cognition . San Francisco: 
W.H. Freeman and Company., 1975. 



Ryan, D.G., Frederiksen, N. "Petformance Test of Educational Achievement. 11 , 
In Lindquist, E.F., (Ed) Educational Measurement . Washington, D.CT: 
American Council on Education. 1951 • * 

Sternberg, R.J. "Intelligence as Thinking and Learning Skills. 11 Educational 
Leadership Vol. 39, No. 1, October 1981, p. 18-20. 



